Application No. 10/726,334 

Response to Office Action of April 4, 2008 



Docket No.: 36761 8016US1 



REMARKS 

Overview 

Claims 1-40 were pending when the Office Action was mailed {April 4, 2008). 
Applicants herein amend claims 1,6, 13, 20, 27, and 35, cancel claims 4 and 5, and do 
not present any new claims. Accordingly, claims 1-3 and 6-40 are currently pending. 

The Office Action rejects claims 35, 36, and 40 under 35 U.S.C. § 102(e) over 
Brunner; rejects claims 1-3, 7-12, 20-22, and 24-26 under 35 U.S.C. § 103(a) over 
Brunner and Vincent; rejects claims 4-6 and 23 under 35 U.S.C. § 103(a) over Brunner, 
Vincent, and Claus; rejects claims 13-19, 27, and 29-34 under 35 U.S.C. § 103(a) over 
Brunner and Plunk; rejects claim 28 over Brunner, Plunk, and Vincent; rejects claims 37 
and 38 under 35 U.S.C. § 103(a) over Brunner and Vincent; and rejects claim 39 under 
35 U.S.C. § 103(a) over Brunner and Plunk. Applicants respectfully traverse these 
rejections. Nevertheless, applicants herein amend the claims to clarify the subject 
matter for which they are seeking protection. 

Applicants' Technology 

Applicants provide a technique for stabilizing an image taken from a video 
camera attached to a moving object (e.g., an airplane) using a combination of electronic 
and mechanical stabilization techniques. Applicants' technique uses an electronic inter- 
frame (or frame-to-frame) stabilization technique to "mov[e] a display area (or viewport) 
within a larger image area" based on the movement of the camera. (Specification, 
[0018]). This inter-frame stabilization may be based on, for example, the velocity and 
orientation of the camera or differences between frames. As an example, a camera 
may capture a 2000x2000 pixel image and only display 1500x1500 pixels (i.e., a 
1500 x 1500 viewport). Because the viewport is 500 pixels smaller than the captured 
image data in each dimension, the viewport can be moved to compensate for "small- 
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amplitude jitter." Applicants' technique also uses the inter-frame stabilization 
adjustment as a basis for adjusting a mechanical line-of-sight controller to compensate 
for "large-amplitude jitter." (Specification, [0018]). In this manner, the inter-frame 
stabilization adjustment is used as a basis for stabilizing an image both mechanically 
and electronically. 

Applied References 

Brunner is directed to a camera system for tracking a target from an aircraft. 
Brunner generates pointing data based on the current position of the aircraft and the 
current position of the target and provides this information to the camera in order to 
"track a moving or dynamic target 14b continuously in real time." (Brunner, 1:29-41, 
2:26-27). Brunner's "pointing parameters may include an azimuth parameter and an 
elevation parameter" and "are based on the adjusted latitude and longitude of the 
aircraft and the known latitude and longitude of the target." (Brunner, 6:21-25). A motor 
assembly then moves the camera based on the pointing parameters "to maintain the 
target 14 along the optical axis 0 of the camera." (Brunner, 6:33-35). 

Vincent is directed to a system that captures image data and simultaneous 
camera position and orientation data and uses this data to generate a three-dimensional 
display of the captured image data. (Vincent, Abstract). As a camera is re-positioned 
and/or re-focused, Vincent's technique records "acceleration in the X, Y and Z direction 
as well as. ..the pitch, roll, and yaw of the camera and [] the distance to the subject." 
(Vincent, 4:18-21). Vincent later uses this data to determine "the precise position of the 
camera, its precise orientation, and the position of the subject" in order to generate a 
three-dimensional image reconstruction. (Vincent, 3:51-54, 4:26-27). 

Claus provides a technique for digitally stabilizing images captured using a 
charge-coupled device ("CCD") attached to a moving or flying carrier. Claus uses 
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inertial sensors to detect the movement around the pitch and roll axes of the carrier and 
corrects image data to compensate for the detected movement. To compensate for 
errors caused by the detected movement, Claus may displace, omit, or duplicate image 
lines and interpolate neighboring pixels to maintain stabilization accuracy. (Claus, 
Abstract, 1:56-61, 2:35-41). For example, to correct for movement around the "roll 
axis," Claus displaces image lines transversely to the direction of flight. As another 
example, to correct for movement around the "pitch axis," Claus omits or duplicates 
whole lines and interpolates missing information using neighboring pixels. 

Plunk is directed to a technique for converting forward looking video into 
downlooking image data. (Plunk, Abstract). Plunk captures video image data from an 
aircraft as the aircraft flies over known ground features. (Plunk, 1:40-43). "After the 
flight, the film is processed to obtain digitized photographs" which Plunk analyzes to 
determine changes between known ground control points in consecutive frames. 
(Plunk, 1:44-45, 54-57). These differences are used to "provide optimal estimates of 
the position and attitude of the aircraft with respect to each frame of imagery." (Plunk, 
2:1-3). Using the estimated position and attitude information associated with each 
image, Plunk generates a top-down view by warping imagery onto elevation data. 
(Plunk, Abstract). 

Rejections under 35 U.S.C. § 102(e) 

Claim 35 now recites "calculating inter-frame stabilization adjustments to account 
for velocity of the vehicle, the inter-frame stabilization adjustments used to electronically 
move the viewport from one frame to the next frame; moving the viewport in accordance 
with the calculated inter-frame stabilization adjustments" and "calculating line-of-sight 
adjustments for the line-of-sight controller based on the inter-frame stabilization 
adjustments." The relied-upon portions of Brunner fail to teach or suggest moving a 
viewport in accordance with the calculated inter-frame stabilization adjustments from 
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one frame to the next. Instead, Brunner attempts to maintain image stability by 
adjusting the pointing parameters of a camera based on position information. In 
contrast, applicants' claimed technology maintains image stability by combining a 
mechanical line-of-sight adjustment with a method for stabilizing an image electronically 
by moving a viewport in accordance with an inter-frame stabilization adjustment. 
Accordingly, claim 35 is patentable over Brunner, as are its dependent claims 36-40. 

Rejections under 35 U.S.C. § 103(a) 

Claim 1 now recites "calculating an inter-frame stabilization adjustment based on 
the velocity and orientation of the transport mechanism, the orientation of the camera, 
the scan and tilt rate of the camera, and distance to the object, the inter-frame 
stabilization adjustment for adjusting the position of a displayed area of an image," 
"adjusting the position of the displayed area of the image based on the inter-frame 
stabilization adjustment," "calculating a iine-of-sight adjustment for the line-of-sight 
controller based on the inter-frame stabilization adjustment," and "controlling the line-of- 
sight controller in accordance with the calculated line-of-sight adjustment." The Office 
Action relies on Brunner at 3:47-64 and 6:11-25 and Claus at 1:24-30 and 2:23-27 as 
disclosing these features. Applicants respectfully disagree that the combination of 
Brunner and Claus discloses these features. The relied-upon portions of Brunner 
describe techniques for redirecting a camera based on the position of the camera and a 
target. The relied-upon portions of Claus describe techniques for digitally stabilizing an 
image based on readings from inertial sensors by displacing, omitting, and interpolating 
image lines to maintain stabilization accuracy. Thus, the combination of Brunner and 
Claus provides an aircraft camera image stabilization system in which the line of sight of 
the camera is dependent on the position of the target and aircraft while digital 
stabilization is based on the pitch and roll of a vehicle to which the camera is attached. 
In other words, Brunner's line of sight adjustment and Claus' digital stabilization are 
based on different data. In contrast, applicants' technology stabilizes a camera image 
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by calculating an inter-frame stabilization value and uses this value to stabilize an image 
electronically and to adjust the mechanical line-of-sight controller for the camera. The 
combination of Brunner and Claus fails to teach or suggest using the same data (i.e., an 
inter-frame stabilization adjustment) to electronically adjust an image and adjust a line- 
of-sight controller for a camera capturing the image. Vincent fails to cure these 
deficiencies. Accordingly, claim 1 is patentable over the applied references, as are its 
dependent claims 2-3 and 6-12. 

Claim 20 now recites "an electronic stabilization component that provides frame- 
to-frame image stabilization based on the specified line-of-sight adjustment rate by 
adjusting the display of an image to remove small-amplitude jitter and that provides to 
the mechanical line-of-sight controller a new line-of-sight adjustment rate derived from 
an amount of frame-to-frame image stabilization to account for large-amplitude jitter." 
The Office Action relies on the rejection of claim 1 in rejecting claim 20. The 
amendments to claim 20 clarify that the electronic and mechanical stabilization 
techniques are based on the same data. As discussed above with respect to claim 1 , 
the combination of the relied-upon portions of Brunner, Claus, and Vincent fails to teach 
or suggest adjusting an image electronically and adjusting a mechanical line-of-sight 
controller for a camera capturing the image based on the same data. Accordingly, claim 
20 is patentable over the applied references, as are its dependent claims 21-26. 

Claim 13 now recites "calculating an inter-frame stabilization adjustment based 
on the determined difference; adjusting the display of the image based on the inter- 
frame stabilization adjustment to remove small-amplitude jitter; calculating a line-of-sight 
adjustment for the line-of-sight controller based on the inter-frame stabilization 
adjustment to account for large-amplitude jitter; and controlling the line-of-sight 
controller in accordance with the calculated line-of-sight adjustment." The relied-upon 
portions of Brunner and Plunk fail to disclose adjusting an image based on an inter- 
frame stabilization. Brunner maintains image stability by adjusting the position of a 
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camera based on the position of a vehicle to which the camera is attached and the 
position of a target. Plunk identifies differences between frames to determine the 
"position and attitude of the aircraft with respect to each frame of imagery." (Plunk, 2:2- 
3). Plunk uses this information to make a "smooth determination of the orientation of 
the camera at the moment of exposure of each frame" prior to warping the captured 
images onto elevation data to generate a top down view. (Plunk, 2:4-5, 19-20). Plunk 
relies on the differences between frames to determine position and attitude, not to 
stabilize images of an object captured from a moving camera. Furthermore, the 
combination of the applied references fails to teach or suggest adjusting an image and a 
line-of-sight controller based on the same data (i.e., an inter-frame stabilization 
adjustment). Accordingly, claim 13 is patentable over the applied references, as are its 
dependent claims 14-19. 

Claim 27 now recites "an electronic stabilization component that provides frame- 
to-frame image stabilization based on a location of an object within the images by 
adjusting the display of the images to remove small-amplitude jitter and that provides to 
the mechanical line-of-sight controller a new line-of-sight adjustment rate derived from 
an amount of frame-to-frame image stabilization to account for large-amplitude jitter." 
The Office Action relies on the rejection of claim 13 in rejecting claim 27. The 
amendments to claim 27 clarify that the electronic and mechanical stabilization 
techniques based on the same data. As discussed above with respect to claim 13, the 
combination of the relied-upon portions of Brunner and Plunk fails to teach or suggest 
using the same data (i.e., frame-to-frame image stabilization) to electronically adjust an 
image and to adjust a mechanical line-of-sight controller for a camera capturing the 
image. Accordingly, claim 27 is patentable over the applied references, as are its 
dependent claims 28-34. 
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Conclusion 

In view of the foregoing, applicants submit that the pending application is in 
condition for allowance and respectfully request reconsideration of the application and a 
prompt Notice of Allowance. If the Examiner has any questions or believes a telephone 
conference would expedite prosecution of this application, the Examiner is encouraged 
to contact the undersigned at (206) 359-8000. 

Please charge any underpayment or credit any overpayment to our Deposit 
Account No. 50-0665, under Order No. 36761 801 6US1 from which the undersigned is 
authorized to draw. 

Dated: Ock. Z, Respectfully submitted, 




Registration No.: 33,273 
PERKINS COIE LLP 
P.O. Box 1247 

Seattle, Washington 98111-1247 
(206) 359-8000 
(206) 359-7198 (Fax) 
Attorneys for Applicant 
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